Intelligent Process Supervision Using Renforcement Learning and Temporal Abstraction

Supervisory control usually involves timely switching among different courses of action over multiple time scales. In this work, intelligent process supervision is addressed in the context of semi-Markov decision processes and reinforcement learning. Temporally extended actions that represent a way of behaving together with a termination condition are used to achieve a set of operational goals/sub-goals comprising a supervision task. The control strategy resorts to a hierarchy of macro-actions or options which are made up of closed-loop sequences of low-level, primitive actions. Supervisory control of a buffer tank is discussed as a representative example. Copyright © 2002 IFAC